Repository navigation
tx: reject page ids at or above the high-water mark in Tx.page - #1346
tony1233200 wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: tony1233200 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
AynSiddiqui
left a comment
There was a problem hiding this comment.
I had asked about this guard on #1318 and wrote the same fix locally, so I ran this branch (TestTxPageBoundsCheck and go test -short . ./internal/... both pass). A few suggestions from my version:
-
Boundary test. Both fixtures have ids far above the high-water mark (137438953473 vs 4, 268435456 vs 6), so
>=vs>is not exercised. A unit test that callstx.page(hwm)andtx.page(hwm+100)on a fresh DB covers it without a fixture. Setting the uint64 at byte 16617 of bbolt_node.go_363.db to 6 gives an id == hwm case; in my build on Nov 14, that one was a SIGBUS in Page.FastCheck before the guard, because the file is 6 pages and the mmap is 8. -
Exact message.
require.PanicsWithValue(t, fmt.Sprintf("pgid (%d) above high water mark (%d)", id, hwm), ...)is stricter than Contains and would catch a wrong hwm in the message. -
Write-tx path. The guard sits after the tx.pages lookup on purpose. A small test that does
tx.allocate(1)in a write tx and thentx.page(p.Id())would fail if someone later moves the check above that lookup. -
Inlining. Putting the same check in DB.page raised its inline cost from 16 to 99 (budget 80) in my build on Nov 14, so Tx.page is the better place. If you have numbers from
bbolt benchor benchstat it would help to include them in the PR body. -
The DCO check shows action required; it needs a Signed-off-by on the commit.
Happy to share the unit tests if useful.
Page ids read from the file were used to index straight into the mmap without a bounds check. A corrupt id either panicked with an index-out-of-range error or, worse, faulted with an unrecoverable SIGSEGV when the bogus page was freed during spill. Check the id against tx.meta.Pgid() after the dirty-page lookup, panicking with a clear, recoverable message like the existing high-water checks in node.go and bucket.go. Tx.Check already recovers such panics into errors. Fixes etcd-io#1318 Signed-off-by: tony1233200 <127579663+tony1233200@users.noreply.github.com>
c3da8c7 to
013200a
Compare
|
@AynSiddiqui Thanks for running the branch and for the detailed suggestions — addressed in 013200a:
Test results: |
|
Thanks — all five suggestions applied in 013200a:
Verified locally: |
|
Ran |
tx: reject page ids at or above the high-water mark in Tx.page
Fixes #1318
Root cause
Tx.pageused a page id read from the file to index straight into themmap (
DB.page) with no bounds check. A corrupt id therefore failed inone of two bad ways, depending on which caller reached it first:
cursor walk:
panic: runtime error: index out of range [562949953425408]in
DB.pageviaTx.page→Cursor.First;copied into
node.pgid, and the nextspillpasses it throughTx.pageintoDB.page, producing an unrecoverableSIGSEGV(
fatal error: faultinPage.FastCheck), not a panic.Fix
After the dirty-page lookup in
Tx.page, panic with a clear,recoverable message when
id >= tx.meta.Pgid(), matching the existinghigh-water checks in
node.goandbucket.go.Tx.Checkalreadyrecovers such panics into errors, so checks on corrupt files now report
instead of crashing.
This does not cover a corrupted meta page whose
pgiditself pointspast the end of the file, as noted in the issue.
Reproduction (before the fix, on main at
a1121b9)Standalone program from the issue, run against the attached reproducer
files (
bbolt_db.go_1055.db,bbolt_node.go_363.db):panic: runtime error: index out of range [562949953425408] with length 281474976710655atDB.pageviaTx.page→Cursor.Firstfatal error: fault/[signal SIGSEGV]atPage.FastCheckviaTx.page→node.spillAfter the fix both cases panic with
pgid (<id>) above high water mark (<hwm>)instead.Tests
TestTxPageBoundsCheckintx_page_bounds_test.go, using theissue's reproducer files as
testdata/issue1318/:WalkWithOutOfRangeBranchElementandPutIntoInlineBucketWithBogusPageId. Both fail without the fix (walk:wrong panic message; put: test process dies with SIGSEGV) and pass
with it.
go test -count=1 -short . ./internal/...: all packages pass.gofmtclean;go vet ./...shows only the pre-existingdb_test.go:1507reflect.ValueOfcopies-lock warning, untouched bythis change.
This PR was prepared with AI assistance.
BoundaryIdsOnFreshDB: on a fresh DB (hwm 4),tx.page(hwm-1)resolves whiletx.page(hwm)andtx.page(hwm+100)panic with the exact message — pins the guard as>=.AllocatedPageInWriteTx: a freshly allocated page is served back from the tx's dirty pages.require.PanicsWithValue.